Remote Object Detection in Cluster-Based Java

Our work combines Java compilation to native code with a run-time library that executes Java threads in a distributed-memory environment with true parallelism. This approach is implemented within the Hyperion system for the distributed execution of compiled Java programs on clusters of PCs. To provide the illusion of a shared memory to Java threads, Hyperion has been built on top of DSM-PM2, a portable implementation platform for multithreaded distributed-shared-memory protocols. We have designed, implemented and experimented with two alternative consistency protocols compliant with the Java Memory Model. The protocols have different mechanisms for access detection: the first one uses explicit locality checks, whereas the second one is based on page faults. We illustrate the effects of the two access-detection techniques with five applications run on two clusters with different interconnection networks: BIP/Myrinet and SISCI/SCI.Notre travail étudie la compilation de programmes Java vers du code natif, en présence d'une bibliothèque run-time permettant des threads Java de s'exécuter en environnement à mémoire distribuée avec du vrai parallélisme. Pour fournir aux threads Java l'illusion d'une mémoire partagée, notre système de compilation '(appelé Hypérion) a été conçu au-dessus de DSm-PM2, une plateforme portable qui facilite l'implémentation de protocoles de cohérence multi-thread pour des systèmes à mémoire distribuée virtuellement partagée. Nous avons conçu, implémenté et expérimenté deux protocole de cohérence alternatifs, compatibles avec le modèle de mémoire Java (Java Memory Model). Les protocoles utilisent des mécanismes différents pour la détection d'accès aux objets partagés. Nous illustrons les effets des deux techniques de détection d'accès à l'aide de cinq applications exécutées sur deux grappes comportant des réseaux d'interconnexion différents : BIP/Myrinet et SISCI/SC

Enhanced Failure Detection Mechanism in MapReduce

The popularity of MapReduce programming model has increased interest in the research community for its improvement. Among the other directions, the point of fault tolerance, concretely the failure detection issue seems to be a crucial one, but that until now has not reached its satisfying level. Motivated by this, I decided to devote my main research during this period into having a prototype system architecture of MapReduce framework with a new failure detection service, containing both analytical (theoretical) and implementation part. I am confident that this work should lead the way for further contributions in detecting failures to any NoSQL App frameworks, and cloud storage systems in general

Extension de la plate-forme DSM-PM2 pour le support de protocoles de cohérence relâchée multithreads

International audienceDans leur présentation traditionnelle, les bibliothèques de gestion de mémoire distribuée virtuel- lement partagée (MVP, en anglais DSM) [8, 11, 12, 4] permettent à des processus de partager un espace d'adressage commun selon un modèle de cohérence fixé. L'objectif du projet DSM-PM2 est de fournir au programmeur d'application distribuée multithread une plate-forme d'implémen- tation où il puisse développer et optimiser conjointement son application et le protocole de co- hérence MVP qui la supporte, de manière portable. DSM-PM2 est actuellement disponible sur des grappes de PC sous Linux, avec les réseaux Ethernet, Myrinet et SCI, et les interface de communication TCP, MPI, BIP, SISCI, VIA, etc. DSM-PM2 fournit les briques de base pour la construction d'une large classe de protocoles de cohérence utilisables dans un environnement d'exécution multithread : il généralise donc les fonctionnalités de MVP comme DSM-Threads [9] et Millipede [5]. À partir de ces briques, 6 protocoles de cohérence sont déjà construits dans la version actuelle. L'utilisateur peut faci- lement les modifier ou en ajouter d'autres. Dans cet article, nous décrivons la mise en place sous DSM-PM2 des deux protocoles de cohérence relâchée multithreads et un aperçu de leurs performances

Enabling Lock-Free Concurrent Fine-Grain Access to Massive Distributed Data: Application to Supernovae Detection

We consider the problem of efficiently managing massive data in a large-scale distributed environment. We consider data strings of size in the order of Terabytes, shared and accessed by concurrent clients. On each individual access, a segment of a string, of the order of Megabytes, is read or modified. Our goal is to provide the clients with efficient fine-grain access the data string as concurrently as possible, without locking the string itself. This issue is crucial in the context of applications in the field of astronomy, databases, data mining and multimedia. We illustrate these requiremens with the case of an application for searching supernovae. Our solution relies on distributed, RAM-based data storage, while leveraging a DHT-based, parallel metadata management scheme. The proposed architecture and algorithms have been validated through a software prototype and evaluated in a cluster environment

An Efficient and Transparent Thread Migration Scheme in the PM2 Runtime System

International audienceThis paper describes a new iso-address approach to the dynamic allocation of data in a multithreaded runtime system with thread migration capability. The system guarantees that the migrated threads and their associated static data are relocated exactly at the same virtual address on the destination nodes, so that no post-migration processing is needed to keep pointers valid. In the experiments reported, a thread can be migrated in less than 75μs

Enabling JXTA for High Performance Grid Computing

Grid computing has recently emerged as a response to the growing demand for resources (processing power, storage, etc.) exhibited by scientific applications. However, as grid sizes increase, the need for self-organization and dynamic reconfigurations is becoming more and more important. Since such properties are exhibited by P2P systems, the convergence of grid computing and P2P computing seems natural. However, using P2P systems (usually running on the Internet) on a grid infrastructure (generally available as a federation of SAN-based clusters interconnected by high-bandwidth WANs) may raise the issue of the adequacy of the P2P communication mechanisms. This paper evaluates the communication performance of the JXTA P2P library over SANs and WANs, for both J2SE and C bindings. We analyze these results and we evaluate solutions able to improve the performance of JXTA on such grid infrastructures

BlobSeer: How to Enable Efficient Versioning for Large Object Storage under Heavy Access Concurrency

To accommodate the needs of large-scale distributed P2P systems, scalable data management strategies are required, allowing applications to efficiently cope with continuously growing, highly dis tributed data. This paper addresses the problem of efficiently stor ing and accessing very large binary data objects (blobs). It proposesan efficient versioning scheme allowing a large number of clients to concurrently read, write and append data to huge blobs that are fragmented and distributed at a very large scale. Scalability under heavy concurrency is achieved thanks to an original metadata scheme, based on a distributed segment tree built on top of a Distributed Hash Table (DHT). Our approach has been implemented and experimented within our BlobSeer prototype on the Grid'5000 testbed, using up to 175 nodes

The Edge, the Cloud and the Supercomputer: Welcome to the Age of the Digital Continuum!

International audienceAvec la croissance spectaculaire de l'Internet des objets (IoT - Internet of Things), le traitement sur des dispositifs connectés en périphérie (edge) est apparu comme un moyen pertinent de décharger le traitement et l'analyse des données des clouds centralisés vers ces appareils qui servent de sources de données (souvent dotés de certaines capacités de traitement). Cela conduit à de nouveaux défis dans les façons de répartir le traitement à travers les infrastructures cloud, edge ou hybrides cloud/edge. L'image complète est en fait plus grande, les appareils IoT et les clouds sont des pièces d'un plus grand puzzle comprenant aussi les supercalculateurs les plus puissants, dans ce qui est maintenant appelé le « Continuum numérique » (digital continuum).Cet exposé portera sur l'émergence de ce terme, sur ses motivations et défis associés. Il traite de la convergence continue des concepts et des technologies sous-jacentes, aux frontières de plusieurs domaines, dont l'informatique distribuée, l'analyse Big Data, le calcul haute performance et l'intelligence artificielle. [Vidéo en ligne]</a

Going Large-scale in P2P Experiments Using the JXTA Distributed Framework

The interesting properties of P2P systems (high availability despite node volatility, support for heterogeneous architectures, high scalability, etc.) make them attractive for distributed computing. However, conducting large-scale experiments with these systems arise as a major challenge. Simulation allows to model only partially the behavior of P2P prototypes. Experiments on real testbeds encounter serious difficulty with large-scale deployment and control of peers. This paper shows that using an optimized version of the JXTA Distributed Framework (JDF) allows to easily deploy, configure and control P2P experiments. We illustrate these features with sample tests performed with our JXTA-based grid data sharing service, for various large-scale configurations

Pyramid: A large-scale array-oriented active storage system

International audienceThe recent explosion in data sizes manipulated by distributed scientific applications has prompted the need to develop specialized storage systems capable to deal with specific access patterns in a scalable fashion. In this context, a large class of applications focuses on parallel array processing: small parts of huge multi-dimensional arrays are concurrently accessed by a large number of clients, both for reading and writing. A specialized storage system that deals with such an access pattern faces several challenges at the level of data/metadata management. We introduce Pyramid, an active array-oriented storage system that addresses these challenges and shows promising results in our initial evaluation